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ABSTRACT 

In November of 1975, a committee was convened at 
Lockheed Palo Alto Research Laboratory to examine the utility of 
computerized data base searching in the public library setting. 
Participants reviewed an on-going computer retrieval experiment 
(DIALIB) being conducted in four San Francisco Bay Area public 
libraries. Librarians reported that since the cost of searches had 
been assumed by the client, the number of searches had declined 
dramatically. Data are provided on search times and client 
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activities and a full report on a cost analysis of initial stages of 
the project* (EHH) 
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SUMMARY 

OVERSIGHT COMMITTEE MEETING 

The major event of the quarter was the Oversight Committee meeting in November. 
Some of the recommendations of the Committee concerned the provision of better 
service access, the continued collection of time and cost data, the need for planning 
for the third year of operation, and the need for NSF support of follow-on studies of 
nonusers. An important recommendation was that the NSF grant support to two 
terminals for the third year, as requested by the CIN libraries, but some concern 
was expressed that continued external funding might delay a full assessment of com- 
puterized search in the public library. 

UTILIZATION OF THE SYSTEM 

Search activity increased during this quarter. During the months of June, July, and 
August, 117 searches were performed, totaling 25 hours of online time, while during 
the months of September, October, and November, 181 searches were performed 
totaling 40 hours of online time. It should be noted, however, that the November 
activity was slightly less than the previous month, 30 it is possible that we are reach- 
ing a steady-state condition on demand. No problems were experienced in the fee-for- 
service mechanics. Most of the searches continue to be custom, rather than standard, 
searches. 

EVALUATION EFFORT 

Because only a relatively small number of questionnaires from the fee period has been 
returned at the time of this report, it is not possible to make comparative evaluations 
between the free and fee period. The results for the questionnaires returned to date 
are given in Section 2. It is hoped that a comparative evaluation can be given in the 
next quarterly report. 
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PUBLICITY 



Newspaper advertisements were tried during this quarter, but the ads produced very 
little patron response. An intensive mailing to one of the communities having a large 
number of professionals was made, and the results are now being evaluated. 

NEW PARTICIPANTS 

At the request of the National Science Foundation, four new libraries able to deal 
with the full cost of search have been selected and will be provided with free terminals, 
a block of demonstration time, and free training for a 6-month period. These libraries 
are the Minneapolis Public Library (INFORM System), the Cleveland Public Library 
(Facts for a Fee), the Houston Public Library, and the Long Island Library Resources 
Council. At the time of this report, all the libraries have received terminals and 
training, and are now in operation. 

Kuiuls have boon requested from the NSK for the evaluation of online search in these 
libraries, and Applied Communication Research has prepared questionnaires for these 
libraries. If these funds become available, we hope to present preliminary data con- 
cerning these libraries in the next quarterly report. 
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Section 1 * 
OVERSIGHT COMMITTEE MEETING 



The final Oversight Committee meeting was held on November 24, 1975, at the 
Lockheed Palo Alto Research Laboratory. Committee members attending were Mr. 
Forrest Carhart (METRO), Prof. Albert Rubenstein (Northwestern University), Prof. 
Charles Bourne (University of California, Berkeley), and Mr. Douglas Fergusen, 
(Stanford University). 

The main theme of the meeting was: 

'Is computerized search useful in a .public library setting, and if so, how can it be 
financially supported ?" The Oversight Committee addressed the following questions: 

(1) What objectives of the original proposal remain to be realized, and is 
it still possible to do something about such objectives? 

(2) • Can we now identify information gained in the study that is transferrable 

to public library groups? 

(3) Are there any new investigations that should be performed in the final 
period of the study? 

(4) What plans should be made for the third year of full-cost service for the 
CIN libraries? 

Reports were presented by the head librarians, reference librarians, our evaluation 
_ subcontractor, Applied Communication Research, Inc. (ACR), and our publicity 
consultant, Mrs. Evelyn Helmer, as summarized in the following sections. The 
Oversight Committee recommendations in response to some of the above questions 
are given in section 1.5. 



ERIC 



1. 1 REPORT OF THE HEAD LIBRARIANS 

Barbara Campbell, Santa Clara County Librarian, and Karl Vollmayer, Redwood 
City Librarian, reported on a meeting of the heads of the participating libraries held 
in November to discuss the third year of computerized search in the public libraries. 
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Attending were Karl Vollmaycr, Redwood City Public Library, Barbara Campbell, 
Santa Clara County Library, Homer Fletcher, San Jose Public Library, and James 
Buckley, San Mateo County Library. Representing the CIN Board were Donald 
Fuller, Santa Clara Public Library, and Patricia Bergsing, Burlingame Public 
Library. 

The main topic was whether to continue computerized retrieval in the public libraries, 
and if so, how to support this operation. It was decided that two terminals would be 
adequate to serve patrons in the Santa Clara County and San Mateo County geographical 
area. Because of severe budget pressures, it was decided to request that the rental 
costs of these terminals for the third year be funded by the* National Science Foundation. 
The full search costs would be passed on to the patron. This request for terminal 
funding was to be presented at the forthcoming Oversight Committee meeting. 

1.2 REFERENCE LIBRARIAN REPORTS 

The reference librarians reported that the mechanics of payment are working out 
satisfactorily, and that there has been little or no complaining on the part of the 
patrons as far as the fee is concerned. The librarians find that they are spending 
more off-line time preparing search strategies now than in the free period. 

The specific comments include the following: 
• San Jose (Charlotte Sakai) 

TM 

The San Jose Public Library is now using an appointment system for DIALOG 
searches and the patron attends the search session about 90% of the time. 
They are still getting a large number of searches from university students, 
and ERIC and NTIS are used most of the time. Five trained searchers are 
available, and all the searchers search all data bases. However, each of the 
searchers specializes in several of the bases so as to be available as a 
resource person when problems arise. A large number of bibliographic 
searches (30 to 40) is being performed each month, with about 75% success. 
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• Redwood City (Lisa Naefl 

One of the biggest problems is the out of county search volume: only 13% 
of the users are from Redwood City, and 50% of the users are from outside 
of San Mateo County. The drop-in approach has been abandoned and all 
searches are done by appointment now. There are five trained searchers 
available, and it is estimated that 45 minutes to 1-1/2 hours are spent in 
discussing the search with the patron. Ms. Naef estimates that most of 
their users are willing to spend $15 per search. 

• Santa Clara County, Cupertino Branch (Lois Thomas) 

It is estimated that 50% of the users are professionals, and some of these 
users are able to use the thesaurus by themselves to determine appropriate 
search terms. Ms. Thomas estimates that users would be willing to spend 
$25 to 50 per search. 



• San Mateo County (Ann Scott) 

Tho search volume is still very low; they are Retting only one search per 
month forwarded from branch libraries, and about three filed directly. A 
new communications link, part of the San Mateo County system, enables 
the library to use the telephone line and the TWX machine, thus cutting 
down on line charges. 

1.3 EVALUATION REPORT 

The report of Applied Communication Research was in two parts. The first part 
presented tho material given in Section 2, and the second part discussed the Cooper- 
DcWuth time study described briefly in the previous quarterly report, and presented 
in its entirety in Appendix B. Briefly, a cost analysis of 411 online bibliographic 
search requests was conducted in four public libraries. For the free period during 
which the study was conducted, the average search cost was $28*41, including both 
personnel and computer time, but not including communication or terminal cost. 
Because the searcher characteristics have changed from the free to the fee period 
(e.g. , from an average of 23 minutes to 18 minutes online time per search), one of the 
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recommendations of the Oversight Committee concerns continuing the Cooper -DeWath 
study into the fee period. 

1.4 PUBLICITY REPORT 

Our publicity consultant, Evelyn Helmcr, presented the material given in Section 3. 
She also pointed out that a major problem exists in directing the patron to both manual 
and computerized search because the first point of contact of many patrons is a 
clerical person at the desk. Clerical personnel often turn away potential reference 
questions due to lack of knowledge of manual and computerized search and therefore 
some way of training or briefing such people on online retrieval capabilities must 
be found. 

Descriptive material must also be developed~to explain the online system to th6 patron. 
At the present time, the reference librarians spend too much time explaining DIALOG 
to a patron attending a search. 

1.5 OVERSIGHT COMMITTEE RECOMMENDATIONS 

After the morning session, at which the reports just described were presented, the 
Oversight Committee met privately in the afternoon. The Committee statement that 
was issued is as follows: 

"The Oversight Committee recommends: 

1. Providing service access, e.g. , reference interviews, at branch and 
other service points most convenient to the user, including exploration of 
ways to provide mail and phone access, to increase the speed and re- 
sponsiveness of the service. 

2. Collection and analysis of time and cost data during the second project ' 
year as essential for effective management and pricing decisions. Data 
should be comparable to that collected by Cooper and DeWath and analyses 
of these data will be an important part of the project's final report. 
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3, Completion by 1 March 1976, of intensive planning for a third year of 
operation in at least the following areas: organization and management, 
staffing and training, target populations and user eligibility, financial 
support and pricing structure, integration into regular library service, 
and standards for speed of service, 

4, External funding of leasing costs for two terminals for a third year of 
experimental use,* This funding should be contingent on the libraries 
continuing to collect cost, time, and user data comparable to the data collected 
by Prof, Cooper and Ms, DeWath, The data will be sent to Lockheed for 
analysis and dissemination as resources are available, 

5, National Science Foundation support for follow-on studies of nonusers 
in potentially high-use target populations, such as: local government 
agencies, small high-technology firms, professional groups, and com- 
munity service agencies, 

6, Presentation of Applied Communication Research reports in a form that 
highlights conclusions, implications, and unanswered questions so that the 
results can better contribute to decision-making in libraries and search 
service organizations* u 

Forrest Carhart 
Douglas Ferguson 
Albert Rubenstein 
Charles Bourne 

1,6 FINAL MONOGRAPH 

The Oversight Committee also discussed the final report monograph. It was deemed 
important that a review copy be made available to the Committee and to the participants 
prior to final publication. It is therefore planned to produce a first draft by mid-May 1976, 
and to discuss the draft with members of the Committee who will be at the SLA meeting 

♦Mr. Carhart and Mr, Ferguson expressed the concern that continued external funding 
might tend to delay a full assessment of the importance of the service, of needed 
budgetary support, and of the price levels required to support direct and indirect costs, 
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in Denver in June. Their comments, and others received in writing, will be in- 
corporated in the final version to be published in July 1976. 

The final monograph, which will serve as a guideline to public libraries interested 
in computerized reference retrieval, will contain the following material: 

• Goals and description of the study 

• Utilization results during the free and fee periods 

• Evaluation results 

• Publicity material developed 

• Statements by the Oversight Committee and participating^libraries 

• Guidelines for libraries contemplating computerized search 

• Suggestions for future studies 

• Appendix material, as required 
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Section 2 
EVALUATION 

The following evaluation data were collected by Dr. Alice Ahlgren of Applied Com- 
munication Research, Inc. , the evaluation subcontractor, during the period June 1, 1975, 
to October 31, 1975. Because only a small number of questionnaires had 6een re- 
turned at the time of this report, it was not possible to make general comparative 
remarks concerning the fee and free service periods. It is hoped that such evaluation 
can be given in the next quarterly report. 

2.1 LIBRARY OPERATION 

The total number of searches conducted each month dropped noticeably after the 
introduction of service charges. Total searches decreased from 285 in May (the last 
month of the free period) to 28 in June (the first month of the pay period). However, 
the number of searches has increased steadily since June by about 10 additional searches 
per month. By October, therefore, the total number of standard and custom searches 
combined had increased from 28 in June to 71 (Fig 2-1). 

The average search time for standard searches dropped from 19. 63 minutes in June 
to 11. 20 minutes per search in October. The average search time for custom searches 
dropped from 14. G4 minutes per search in June to 12.62 minutes per search in October 
(Fig. 2-2). 

The principal data bases used during this period were ERIC, Psychological Abstracts, 
NTIS, and Engineering Index, in that order. 

Noticeable differences continued between libraries in terms of the period which elapses 
from the time the patron requests a DIALOG search to the time the patron receives 
the search results. San Jose completed all searches in less than a week. Cupertino 
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completed 60% of their searches in 2 to 7 days, and another 30% in 2 to 3 weeks. 
Redwood City completed 75% of their searches in less than 1 week. San Mateo took 
2 to 3 weeks to complete nearly 70% of their searches. Both Cupertino and Redwood 
City arrange appointments for patrons to be present at the search. As these appoint- 
ments are scheduled at the patron's convenience, the time required for the patron 
to receive search results is partly dependent on how soon the patron schedules an 
appointment after the initiation of the search request. In other words, some of the 
delay experienced at the libraries in completing searches is due to the patron and 
not to staff overloading. 

Preliminary data from the pay period indicate that librarians are spending more 
time offline in preparing for DIALOG searches. Although offline search preparation 
time still varies widely between libraries, all library staff indicate that they are 
spending more than 15 minutes on anywhere from 45 to 80% of all searches. During 
the free period, 50 to 80% of all searches took less than 10 minutes of offline prepar- 
ation time. 

2.2 OBTAINING SOURCE DOCUMENTS 

Thirty-one percent of the patrons obtained source documents from the library at 
which the DIALOG search was done, 25% from college libraries, and 14% from branch 
libraries. Other sources included NTIS (8.6%), authors (5.7%), and bookstores, 
publishers, and company libraries. 

2.3 SEARCH COSTS 

Nearly 41% of all searches completed by the libraries cost the patron from 5 to 10 
dollars. Another 20% of the searches cost less than 5 dollars. 

2.4 PATRON CHARACTERISTICS 

Major groups using the system continue to be technical professionals (including 
civil, nuclear, and electronic engineers, geologists, and computer specialists) and 
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graduate students. The next two major sets of users are individuals in the field 
of education (including teachers,, professors, and school administrators) and business 
professionals. Use of the system by college undergraduates has dropped off somewhat 
since fees were initiated. 



The majority of DIALOG searches continue to be done as part of the patron's job 
(40.9%) or for a research paper (34.2%). 

Librarians and friends continue to be the principal sources through which patrons 
hear about the availability of the service. Notices in the library and information 
about the service from college professors are the second most important publicity 
sources. 



2.5 USEFULNESS TO PATRON 



Patron ratings of the value of the DIALOG search to them varied widely between 
libraries. At San Jose and San Mateo, 40 to 50% of DIALOG patrons rated their 
searches as being of major value. At Cupertino and Redwood City, 16 to 20% of 
DIALOG patrons rated their searches as being of major value. Another 40 to 60% 
rated their searches as being of considerable value at all libraries except San Mateo, 
where 16% of the searches fell into that category. 

In addition, 60% of DIALOG patrons indicated that the results of their DIALOG searches 
provided sufficient reference to answer their questions adequately. 

Approximately 14% of the patrons found no citations of use. Another 20% found from 
1 to 5 citations of use, about 25% found from 6 to 20 citations of use, and about 40% 
found more than 21 citations of use. 
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Section 3 



PUBLICITY ACTIVITIES 



The publicity generated since October 1975 by Evelyn Helmer, Publicity Consultant, 
included: 

(1) A booth at the annual meeting of the California County Supervisors Associ- 
ation, featuring a terminal, large posters, brochures, printed material 
and a slide-tape show (November). 

(2) Demonstration of DIALOG and distribution of printed materials at the 
California Library Association meeting (December). 

(3) Four ads were placed in the Redwood City Tribune, November 24 through 
December 4, 1975. These ads appeared on the second page of the newspaper, 
as Shown in Fig. 3-1. (The ads produced very little activity: at the Redwood 
City Library there were two requests, one in music and the other religion. 
Neither were appropriate DIALOG searches. At San Mateo County no 
requests were received that were the direct result of the ads.) 

(4) , Large posters, produced last year, are now being distributed again, with 

tear off sheets at the bottom, see Fig. 3-2. This permits those who want 
further information to write or phone. 

(5) A letter, brochures, and data base list are being mailed to Ladera residents. 
This is a community of approximately 450 families, middle and upper income, 
who are largely professional and therefore a high, potential target area. A 
similar mailing will go to officials listed in the Government Directory for 
planning for Santa Clara County. (Sec Fig. 3-3 for letter.) 

(6) : A 30-second animated cartoon is planned for a TV spot. This to be used on 



as many major networks as will accept it and some cable stations. It 
will be used if the libraries with terminals need additional publicity or can 
cope with it. 



19 



12 




lOCKHMO 



LOCKHEED PALO ALTO RESEARCH LABORATORY 



M I S S I I 1 $ 



S P A C I 



COMPANY. INC. 
All COlPOtATION 



A 



s u • $ i o i a I r 



o I 



IOCRHII0 



LMSC-D464275 



9 

ERIC 



ripir 




'.41 Hi' I 



Tired? Frustrated 

by endless research? 

Save time and money. Ask your 
public library for the services of 
computerized information retrieval. 

For ■ smell fee, bibliographies will bo com- 
elled In f ho f tele's of! 



" Business * Finance * Cloims • Educoiion 

• Psychology * Social Service * Government Reports 

• Chemistry • Agriculture • life Sciences • Engineer- 
ing * Physics * Meteorology * Oceanography 

FAST COMPtm UF-TO-DATI 



Picas* it*. 



librae y C«fitputerli«tf Retrieve!, 23 Tewer 94., ■•Intent 
f 4002 it Cell* Reewtee* City library 3et-3737: 
Sen /Ami— Ceonty library 373-2071 




I I 



I — .-- „___ — r , 

I'lxjr 01?ti/i r 'l«!C , ' : ..Oi )K-av<(.; 



b:VijciKM'cj:jj]l;t-__ 



Featuring Old Fat 
Quality and Crafts; 



Iffltoa Hon 



( 



M pleased to announce it 

mm " 



^ 8 



ri0"l4 4* KM ftf 

M«j «( IMS MM* WO 




fe— Redwood City (Co.) Tribune Thurs., Nov. 20, 1975 
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Fig, 3-1 Newspaper Advertisement Used 



For further information- 
Write: DIALOG, 25 Tower Road , Belmont, CA 94002 



Name 



TTo: Street 5 CTty FTp 

it it it 

Or call: 

Rodwood City (415) 359-3737; 

San Jose (400) 287-2788; 

Other cities in San Mateo (415) 573-2071; 

Other cities in Santa Clara (408) 253-6212. 



Fig. 3-2 Tear-Off Sheets Used on Large Posters 
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COUNTY of SAN MATEO 

SAN MATEO COUNTY FNEE LIRRAHY 

IB TOWER ROAO. BELMONT, CALIFORNIA 04OO2 
(413) 373 203f 

PUBLIC INFORMATION 
♦ (415) 573-2062 

COMPUTERIZED RETRIEVAL SERVICE IN THE PUBLIC LIBRARY 



Are you keeping pice with the Information explosion? More and more 
business and professional people find they can't keep up with the 
vast amount of knowledge that 1s accumulating In all fields. Space 
age technology requires a computer to store and retrieve sources of 
Information. And a computer Is just what Is available In your local 
library today 1 It can find sources of Information In a matter of 
minutes. 

This service Is DIALOG, computerized on-line Information retrieval, 
developed by Lockheed Research Systems, now accessible to all who 
live, work, or attend school In Santa Clara, San Mateo, Santa Cruz, 
and Monterey Counties. This two-year experimental Rroject was made 
possible by a grant from the National Science Foundation. 

WHAT DO YOU RECEIVE WHEN A SEARCH IS CONDUCTED FOR YOU? 

A list of references 1s printed that draws upon a vast store of know- 
ledge' contained 1n more than 20 data banks In the fields of education, 
agriculture, psychology, science, engineering and business. Your 
local library will obtain the books and journals cited on your bib- 
liography, If you wish to use them In your research. 

Some requests that have already been searched on DIALOG are: 
CkUdKcn in hin$tt patent rfomctui, cuUUtic chitdKVt, dance, thviapy, 
htticoptzA noitt ttveJU, ncopKtnt dztvUoKation, 4ilt-actualization, 
cat tcukcmia, induAtniat tooAttb and othvu*. 




IS THERE A CHARGE? 



Until June, 1975, searches were con cted free of charge, under the 
terms of the project grant; now there Is a fee that Is roughly half 
the commercial cost of the service. Please see flyer enclosed. 

If you need help In finding scientific or technical Information, ask 
your local librarian for a DIALOG search. 

LET A COMPUTER DO YOUR WORK! 

V^ry truly yours, 




Evelyn G. Helmer 
Public Information 



Fig. 3-3 Letter Sent to Ladera Residents 
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3.1 SOME TENTATIVE CONCLUSIONS 



Some tentative conclusions by Mrs. Helmen are as follows: 

"Although appealing directly to the public by mailings, distribution of brochures, 
posters, newspaper ads, the most effective publicity for this type of service in 

« 

public library is probably through professional newsletters that focus on a target 
consisting of a high number of potential users, i.e., scientists, engineers, teachers, 
students, etc. 

"Another effective approach is the demonstration and distribution of printed materials 
at appropriate conventions and professional meetings. 

"Data collected by ACR and spots checks by myself indicate that 'word of mouth 1 is 
the real publicity winner. One satisfied'patron tells another. It is slow, but ultimately 
it will produce results. Certainly initial publicity must be provided by traditional 
means: brochures, posters, flyers, and the like. BUT THE ESSENTIAL INGREDIENT 
VOW TUli SUCCIiSS OK PUBLICITY FOR THE DIALIB PROJECT LIES IN EDUCATING 
THE LIBRARIANS AND NONPROFESSIONAL STAFF OF THE PUBLIC, SPECIAL, 
AND ACADEMIC LIBRARIES THAT ARE EXPECTED TO PARTICIPATE IN THE 
PROJECT. The staffs should be thoroughly educated in the capabilities of the DIALOG 
system and efforts made to enlist their enthusiasm for the project. In public libraries 
often a patron's first contact is with a nonprofessional staff member. It is essential 
that these staff members be informed, so they can identify requests and questions 
suitable for a DIALOG search. 

"Hindsight suggests that the project should pay for short presentations to local pro- 
fessional organizations, businesses, and government agencies. These appearances 
might include a short slide or film show, brief talk, then a question and answer period. 
Printed brochures and other materials can then be distributed. Stringent staffing and 
tight budgets allow little time for this in most public, libraries, therefore it should be 
supported by the project. When contacting organizations and agencies for talks or 
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demonstrations it is imperative that direct invitations be sent to middle management 
and other staff who are likely to be engaged in research related activities. Propa- 
ganda sent only to department heads often does not filter down to other members. ,! 

3.2 THE FUTURE 

In the months remaining in the project, publicity will concentrate on radio and TV, 
if this additional publicity is required or acceptable to the four project libraries. 
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Section 4 
MEETINGS AND PRESENTATIONS 



4. 1 RBFKHBNCK LIBRARIAN MEETING 



A meeting of reference librarians was held at Lockheed on October 6, 1975. Some 
of the topics discussed included: 

• Publicity. The various libraries expressed their views concerning the need 
for publicity. 

• Mechanics of billing. Questions concerning the Lockheed bills to the 
libraries were resolved. 

• Evaluation. Some of the ACR evaluation results were discussed. 

• Bibliographic searching. Some recent experience in bibliographic searching 
was indicated. 

• Search critiques. The librarians were reminded of the availability of 
critiqueing service. 

• Offline time. The librarians indicated that more time was being spent 
offline to prepare searches. 

• Time and cost study. The results of the Cooper-DeWath time and cost study 
were presented by Nancy DeWath. 



4.2 PUBLIC PRESENTATIONS 



ERIC 



The following papers, presentations, and demonstrations were given during this 
period: 

• California Supervisers Association meeting, San Jose, October 1975. 

Demonstration searches were performed for several days in the exhibit area. 
1975 Annual Convention of the American Society for Information Science, 
October "Providing Online Search Services Through the Public Library," 

24 
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Alice E. Ahlgren (ACR) u Fee for Online Service in a Public Library 
Setting, 11 Roger Summit and Oscar Firschein. 

• Workshop on Online Bibliographic Systems, ASIS Deleware Valley Chapter 
and Drexel University Graduate School of Library Science, November 7, 
1975, Philadelphia, Pa., u Online Systems in Public Libraries, M Robert 
Donali (Lockheed). 

• California Library Association meeting, San Francisco, December 1975 
Nancy De\Vath(San Mateo County Library) described the project at the 
Library Automation Session. 

• Alice E. Ahlgren completed her thesis, "Cost/Utility Implications of 
Providing Online Search Services Through Public Libraries, 11 Stanford 
University. 
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Section 5 
MISCELLANEOUS ACTIVITIES 



5. 1 NEW PARTICIPANTS IN STUDY 



As indicated in the last quarterly report, at the request of the National Science 
Foundation, public libraries able to bear the full cost of search service were invited 
to participate in the study. These libraries will be given free terminals arid a pool 
of demonstration time for a 6-month period in return for evaluation of the impact 
of full-cost fees on patrons ♦ The libraries selected were: 

• Minneapolis Public Library (INFORM system), Minneapolis, Minn, 

• Long Island Library Resources Council, Bellport, Long Island, N.Y. 

• Houston Public Library, Houston, Texas 

• Clovohtiul Public dJhrary (Knots for a Fee), Cleveland, Ohio 

These libraries have now been provided with terminals,* a block of free demonstration 
time, and free training. They will be charged the full search cost, and will pass these 
costs on to their patrons. 

Additional funding has been requested of the National Science Foundation for evaluating 
the use of online search in a full-cost environment. Although this funding has not 
yet materialized, ACR has prepared questionnaires for these libraries and data col- 
lection has begun. It is hoped that we will be able to provide preliminary evaluation 
results in the next quarterly report. 

5.2 PUBLIC USE OF TERMINALS 

Two of the CIN participating libraries, Santa Clara County (Cupertino), and San Jose, 
have allowed patrons to use the terminals to access local time-sharing computer 
services. This has proved to be a very popular facility. 



ERIC 



♦Except for Long Island Library Resources Council, who are availing themselves of 
a nearby terminal. 
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5.3 USE OF CREDIT CARDS 



The CIN participating libraries have not been interested in using BANKAMERICARD, 
MASTERCIIARGE, or similar charge card services that would allow the patron to pay 
for a search by credit card even though the service charges could be reimbursed as 
part of the study. Part of the reluctance seems to stem from the difficulties of "going 
through the legal and accounting departments of the City or County to set up the service, 
and partly from the fear of damaging the "image" of the public library as a source of 
free service. (It should be noted, however, that several of the libraries have set up 
institutional accounts for repeat users of the system.) 

5.4 SEARCH CRITIQUES 

The participating CIN libraries have not made full use of the critiqueing service 
offered. (They have been offered review* of one search per month per library.) 
During this period, six searches were critiqued, resulting in four DIALIB notes 
describing the problems encountered, and. their solutions. These notes were sent 
to all the participating libraries. 

5.5 ENHANCED TWX TERMINAL 

It should be noted that it is now possible to lease an enhanced TWX terminal that allows 
the user to communicate either by Western Union line or by telephone line, as selected 
by the user. The character rate of the terminal is the usual 10 characters per second 
of a TWX. This terminal leases for only $40 per month more than the usual TWX 
device, and is an attractive way for a public library that already has TWX services 
to obtain access to online retrieval services without providing a separate terminal. 
This terminal is available from RCA, and Western Union will probably have this type 
of terminal also in the near future. 
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Appendix A' 

EVALUATION DATA GATHERED BY APPLIED COMMUNICATION RESEARCH 
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(The following information has been obtained from 
searches completed after the beginning of the pay 
period. ) 



Table of Contents 



Tables 



Number of Searches 
Mean Search Time 
Frequency of Use 

Source Through Which Patron Heard 
About the Availability of DIALOG 
* Occupation of DIALOG Client 

Reason for DIALOG Search 

Sources Through Which Documents Are 
Obtained 

Time (Number of Days) For Patron to 

Receive Citations 
Value of Search to Patron 
DIALOG Question Answered Adequately 

\\y Library 
Number of Citations of Use to Patron 
Cost of DIALOG Search to Patron 
Offline Search Preparation Time 

DIALOG Patron Comments 



Page Number 

1 

2 
3 

4 
5 
5 



7 
7 

7 
8 
9 
9 



10,11,12 



Flppliad [jCfnuuiiaaiici PSasasrafi 



P.O. BOX 5849 



STANFORD 



CALIFORNIA 94305 



9 

ERIC 



29 



/ 




o 

U 

o 



SO 



N K) 
rH tO 



vO <tf 
r— I i— I 



tO rH O O 



<D 
CO 



o cvj lo ro 



CT> v£> rH O 
rH <N3 



O O vO O 



to 



vO O CN1 O 



00 



CO 

w 

X 

u 
< 

CO 



< 



<m o rH lo loo r^r^tor^ 



H N 



o 



rH O | 



00 



o 
a: 



3 

»-5 



tO O tO LO rH 



O CvJ O CvJ 
<N3 



ft 



LO O rH VO 



E3 



0\ O rH O I O 



3 



o 



<nj vO LO VO 



O M 



pi 



LO 





o3 

G 
o3 

+-> 

CO 









>> 






















>*> 










+-> 






















•M 










•H 










•H 












♦H 






o 


o 


(J 






o 


o 










o 


o 


CJ 






c 


<D 










<D 








o 


c 


o 






I/) 


• H 


•M 






c/> 


•H 


+-» 


T5 




o 


l/> 




•M 


T3 




o 


|B j 


iS 


O 




O 


•M 


a: 


O 






o 


•M 


03 


O 


l/> 




r< 




O 


rH 




f-. 




o 


rH 


•M 








O 


rH 




<D 




3: 


o3 




<D 








03 




<D 




% 


03 


c 


a 


CJ 


T3 


+-> 




a 






+J 


J~« 






c 




•M 


o3 




03 


0 


O 


03 




o3 


<L> 


O 


+-> 


03 




03 


0 


O 


CO 


u 


CO 


a; 


H 


£ co 




CO 


a: 


H 


t/> 


CO 




CO 


a: 


H 



o 

£ 



30 



U 
«H 

03 
CO 

o 

rH 

•H 



0 
I/) 
O 
>-> 

03 
CO 



o 



o 

<D 
•M 
03 

s 

03 
CO 



+-> 

CJ 

T3 
C 
O 

a: 



r 



1 




W 



O 
OS 
< 

w 
co 

< 
w 

2£ 





to 


o 


OO 




O 


O 




O 




CM 




vO 


LO 






(SJ 






(NJ 


vO 






• 










• 








• 


o 


CM 


rH 


a\ 


rH 


rH 


UO 




VO 


rH 




p 


rH 


rH 




rH 


rH 


rH 






rH 


rH 


(J 






















o 
















































rH 


uo 


VO 


rH 




UO 


rH 


00 


00 








*f 


ro 


(VJ 


00 


vO 


(NJ 


rH 


rH 




I 
























ro 


o 




00 


UO 


(\j 


l—\ 




(NJ 






I 




i—l 










pH 


pH 














































CO 




















t 








00 




vO 


rH 


UO 


ro 


vO 


r>. 


4_) 








\Q 




<NI 


UO 


o 


00 


00 


</) 


• 


I 
























ro 


00 




O 


ro 


CT) 


O 


ro 


cc 






rH 


rH 


rH 


rH 


rH 


rH 


<N 


rH 
























< 
































ro 








00 




>> 


vO 




rH 


r-i 


vO 








rH 


o 


rH 






















D 








UO 


ro 


00 






UO 






rH 






rH 


rH 




rH 












ro 




vO 


ro 




O 


vO 






a) 


O 








vO 


00 




ro 


00 


vO 


c 


• 




























00 


cn 




00 • 


ro 


ro 








rH 




rH 


rH 




rH 


rH 


rH 


rH 




P 

CO 





















>> - 










p 












p 










♦H 












•H 






O 


o 


U 








O 


o 


U 








a> 








a> 




a> 








•H 


4-> 


XI 






to 


•H 


p 


T> 


• • 


o 


P 


ctJ 


O 


(A 




o 


P 




O 


(A 


^ 






O 


rH 








S 


o 


rH 




0) 




3: 


cvl 






a> 








c 


Pu 




T> 


P 






P. 






P 






cd 


0) 


O 










a> 


o 


CO 


u 


CO 




H 


e 


CO 


U 


CO 




H 



9 

ERIC 



O 
P 

to 

D 
O 



31 



FREQUENCY OF USE OF DIALOG DATA BASES 



Data Base Per Cent of Total* 



ERIC 


36.25 


Psychological Abstracts 


31.5 


NTIS 


25.5 


Engineering Index 


11.4 


ABI 


9.4 


Chemical Abstracts 


6.7 


lNbrfcL 


6. 7 


Social Science Citations 


5.4 


CMA/EMA 


,3.4 


BIOSIS 


3.3 


COMPENDEX 


.2.0 


NAL/CAIN . 


2.0 


F $ S 


2.0 


Exceptional Children Abstracts 


1.3 


AIM/ ARM 


.7 




N=149 


*Percentages do not add up to 100 


because 


searches are often done on more 


than one 


data base. 
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Source Through Which Patron Heard About the 
Availability of DIALOG 



Source Per Cent of Total 

Librarian 24.2% 

Friend 24.2 

Notice in Library 14.8 

Professor 10.1 

College Librarian 10.1 

Newspaper 3.4 

Mailed Notice -7 

Club Meeting .7 

N = 149 



# 

ERJC 
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Occupation of DIALOG Client 



Occupation 

Technical Professionals 
Graduate Students 
Education 

Business Professionals 

Professional (M.D. , Lawyer, 
Psychologist) 

Librarians 

College Students 

Scientist or Researcher 

Social Worker 

Counselor 

Writer 

Government Employees 
Nurses 

Ministers, Priests 
Other 



Per Cent of Total 

18.1% 
16.1 ' 
14 .8 
7.4 

7.3 

6.7 

4.7 • 

4.0 

3.4 

2.7 

2.0 

1.3 

1.3 

1.3 

2.1 

N=149 



Reason for DIALOG Search 



Reason for Search 

Job 

Research Paper 
School Assignment 
Personal Interest 
Advanced Degree 
Book 



Per Cent of Total 

40.9% 
34.2 

6.0 

6.0 

4.7 
.7 

N=149 
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Sources Through Which Documents Are Obtained 



Source Per Cent of Total 



Library At Which DIALOG Search Is Made 31.4% 

College 25.7 

Branch Library 14,3 

NTIS . 8,6 

Authors Directly 5.7 

Stanford 5.7 

Bookstore 2.9 

San Mateo County Office of Education 2,9 

State Mental Health Department 2,9 

Pub! ishcr 2,9 

San Mateo County Health and Welfare Department 2,9 

Company Library 2.9 



N=35 
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Time (Number of Days) For Patron to Receive Citations 

San San 
Number of Days Jose Cupertino Mateo Redwood City 



1 day 


14.3% 


0% 


0% 


16. 


2 - 7 


85.7 


60.0 


33.3 


58.3 


8 - 14 


0 


20.0 


16.7 


25.0 


15 - 21 


0 


10.0 


50.0 


0 


Over 21 


0 


10.0 


0 


0 




N = 7 


10 


6 


12 



Value of Search to Patron 



Value 



San San 
Jose Cupertino Mateo 



Redwood City 



Major Value 


42. 


9% 


20. 


0% 


50. 


0% 


16.7% 


Considerable Value 


» 42. 


9 


60. 


0 


16. 


7 


50.0 


Minor Value 


14. 


3 


10. 


0 


16. 


7 


25.0 


No Vaiue 


N = 7 


0 


10. 
10 


0 


6 


0 


8.3 
12 



"Hid the results of your DIALOG search provide sufficient references 

to answer your question adequately?" 



Answer 



San San Redwood 

Jose Cupertino Mateo City Total 



Yes 
No 



57.1% 
42.9 
N = 7 



60.0% 

40.0 

10 



66.7% 
16.7 

6 



58.3% 
41.7 

12 



60.0% 
37.1 

35 
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Number of Citations of Use to Patron 
(% of total population) * 



patron population 




erJc • 37 



Number 


of Citations 


San 
Josa 


Cupertino 


San 
Mateo 


Redwood 
City 


Total 




0 


01 


8 • 6 % 


0% 


5.7% 


14 . 3 




1 - 5 


5.7 


0 


0 


14.3 


20.0 


6 


- 10. 


0 


0 


8.7 


0 


8.7 


11 


- 15 


2.9 


2.9 


2.9 


0 


8.7 


16 


- 20 


2.9 


0 


0 


5.8 


8.6 


21 


- 50 


0 


11.6 


5.8 


2.9 


20.3 


More 


than 50 


8.7 


5.8 


0 


5.8 


20.3 




N 


- 7 


10 


6 


12 


35 




*f igures 


represent 


per cent 


of total" 


DIALOG 
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Cost of DIALOG Search to Patron 



Cost 



Per Cent of Total 



$0 - 4.99 

5 - 9.99 

lu -14.99 

15 -19.99 

20 -24.99 

25 -29.99 

30 -39.99 

40 -49.99 



19.5% 
40.9 



9 
12 
4 



3.4 
1.3 



0.7 



N=149 



Offline Search Preparation Time 



Time 
(in minutes) 



San Jose Cupertino San Mateo Redwood City 



0 

5 - 10 
15 - 25 
30 - 40 
45 - 60 
Over 60 



N = 



6.5% 
8.7 
47.8 
28.3 
8.7 

46 



33.3% 
20.9 
14.6 
25.1 
4.2 
2.1 

48 



23.1% 

30.8 

15.4 

15.4 

15.4 

13 



11.9% 
11.9 
28.6 
26.2 
14.3 
7.1 

42 
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DIALOG PATRON COMMENTS 



(November, 1975) 



These comments are listed almost verbatim as they were written down 
on the follow-up questionnaires. Comments are listed by the library at 
which the patron had the search done, even though many of the comments 
apply to DIALOG service generally. These comments are from questionnaires 
received after the beginning (June 1, 1975) of the pay period. All comments 
are included in this list. 



SAN JOSE 



"Great service - do everything possible to keep it, 11 

"The information retrieval system obtained references that I could not find 
on my own and these references listed extensive bibliographies which were 
helpful in expanding my topic, I was very pleased with prices, speed and 
results of the DIALOG service. Also, the employees were very helpful," 

"Those systems should be available more widely. It would be a good system 
to replace card catalogs," 



CUPERTINO 



"Should have research student discount, I'm working on master's degree, and 
although a great developer of a basic foundation for bibliography, I can't 
afford further 'narrowing 1 searches," 

"Use of the computer to furnish bibliographic and source material has been 
tlmcsnviiiR, dependable and efficient," 

"Having used DIALOG, I have two questions: (1) Was the information I desired 
in the system? and (2) were the keyoperators and I just not generating the 
appropriate keywords and combinations?" 

"For a simple search the service would be fine. The information I needed was 
too complex. There wasn't sufficient information for recall and if there had 
been I would still have had to spend considerable time selecting and choosing 
so that in the long run I would not have saved any time. The service might have 
worked for one specific compound, but I was surveying a broad series of cottr t iounds , " 
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CUPERTINO (cont.) 



"I live in the area of the Cupertino DIALOG service during the summer; 
it is not available to me during the rest of the year." 

"This is the first time I have used an information retrieval system such 
as DIALOG. My questions were quite general and it therefore makes it 
socnewhat difficult to answer some of the above questions very specifically. 
I foufrd the service to be very useful and am pleased that it is available 
for possible future use. 11 

« 

M I primarily use University (UCB) library since I am a PhD candidate there." 



SAN MATEO 



"When I first read the material on DIALOG, I was under the impression that 
it would supply the citations, papers, reports, etc. to the user. I didn't 
know it would just give a detailed bibliography. On that point, I was 
disappointed." 

"The people whom I dealt with were great. They were very friendly and 
helpful. Thank you." 

"As in the past, the information was up to date; very useful for myself and 
other members of staff with similar interests." 

"Information provided was of great value in research; sp.ecific information 
which provided basis for follow up research. A great help and I will use 
it again. Thanks." 



REDWOOD CITY 



"An exceptional service; extremely helpful to me," 

"Information provided by DIALOG already in my possession thru National Bureau 
of Standards publications." 

"It would be very helpful - necessary, in fact - to leave a Chemical Abstracts 
Thesaurus in the library. I'm certain that if the correct key words could have 
been found, this search would >>ave been more valuable." 

"Not only was the DIALOG search a tremendous help to me, the reference librarian 
(Lisa Naef, Redwood City Public Library) who did the search was extremely 
helpful and very generous with her time and expertise. She was quite skilled at 
making efficient use of the system. Without her expert help in designing my 
search, I'm sure the information retrieved would not have been as appropriate 
and useful." 
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REDWOOD CITY (cont.) 

"There needs to be a better communications link between user and DIALOG 
system. That is - easier source to determine key words, identify topics, 
etc. Generally, "How am I better able to know what DIALOG can provide me? M . 

"Quite pleased with data. My subject will require older references (earlier 
than 1969); however, information retrieved indicated other researchers did not 
have your data/ 1 

"It's still a wonderful service and I hope the price doesn't go up." 
"Rather disappointing results from this particular search. H 
"Useful and effective - a little expensive. 11 
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Appendix B 

REPORT ON THE COST OF ONLINE BIBLIOGRAPHIC SEARCH 
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This paper reports a project commissioned by Applied Communication 
Research, Inc. as part of an on-going program studying potential user 
Interfaces for on-line search services. 

This research program Is funded by the Office of Science Information 
Service, National Science Foundation, through grants to the Lockheed 
Missiles and Space Company (G^2299) and to Temple University (GM2271). 
Applied Communication Research, Inc. serves as an evaluation subcontractor 
to both projects. 

Alice E. Ahlgren Is program mlnager of the ACR evaluation of the 
Lockheed OIALIB project. Colin K. Mick manages the ACR-Temple effort. 

As. noted In this report, the study was conducted during the first year 
of a two-year project — during the "free service 11 portion of the study. 
As a result, the various time estimates have been affected by a number of 
variables. The time estimates presented here may therefore be misleading 
for the following reasons. 

1. The searchers were still In a learning mode — the learning 
curve for searching (shown by time per search) started at 31 
minutes per search and dropped to about 18 minutes at the end 

of the free period (mean for the free period was approximately 23 
minutes) . 

2. Our data from the pay period (which began In June, 1975) 
indicate that the searchers are now becoming more sophisticated 
and tend to devote more time to off-line search preparation 
and less time to actual on-line searching. 

ACR, Lockheed and the participating libraries are now discussing the 
possibility of replicating this study during the pay period. 
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ABSTRACT 



A cost analysis of k\\ on-line bibliographic search requests was 
conducted. The study Involved monitoring the time that 35 Individuals In 
four public libraries spent processing these requests* The study 
Identified a set of seven tasks that are performed for each request and 
determined the ^average time and cost for each of the tasks. The average 
total search cost was $28. 41 exclusive of telephone line charges. This 
figure Included the data base connect charges of $17.29. The average time 
to process a request was 7.8 calendar days. A wide variation In the cost 
and time figures was found among the four libraries and among Individual 
senrchers. 



INTRODUCTION 



On-line bibliographic searching is becoming commonly used as an aid to 
the reference librarian and researcher. ^C^mmerclal ly available systems, 
such as Lockheed's DIALOG and System Development Corporation's ORBIT, 
provide access to a multitude of machine searchable data bases for this 
purpose. Many issues remain unresolved wl th respect to the general process 
of on-line searching. These issues include questions such as the 
effectiveness of on-line searching, methods for training searchers, and 
optimal search strategies at the terminal. This paper examines the cost of 
on-line searching. On-line search costs Include the charges that are 
Incurred for connection to a commercial search service, the cost of 
printing bibliographic citations, and the cost of the reference librarians 1 
time. 

There are a number of reasons for studying the cost of on-line 
searching. In the first place, it seems apparent that on-line searching Is 
a close substitute for some forms of non-computerized bibliographic 
searching. If this Is the case, and If the end product Is the same, then 
It is Important to know how the costs of the alternatives compare. 
Secondly, costs are an important tool to aid In resource allocation. The 
provision of any new service implies either additional funding or a 
diversion of funds from one type of service to another. With cost data 
this type of decision can be aided considerably. Finally, costs can 
provide a basis for making pricing decisions. It seemsqulte likely that 
users will have to pay for on-line bibliographic search services. The 
question Is, how much?. While there are numerous ways In which prices can 
be set (i.e., loss leader, marginal cost, cost recovery), a knowledge of 
costs can play an important part In their establishment. 

The cost data reported In this paper were compiled as part of an 
ongoing project being conducted by the Lockheed Palo Alto Research 
Laboratory under the sponsorship of the National Science Foundation's 
Office of Science Information Service. As part of the study, Lockheed's 
on-line reference retrieval service (DIALOG) is being made available 
through four public libraries In the San Francisco Bay Area. On-line 
search services were provided at no cost to patrons during the first year 
of ' the project (August, 197** through May, 1975). The search service Is 
being provided at a reduced cost to patrons during the second year of the 
project (June, 1975 through May, 1976). 

The four libraries participating in the project are all members of the 
Cooperative Information Network (CIN), a Bay Area consortium of public, 
special, and academic libraries. CIN, In cooperation wl th Lockheed, 
selected four public libraries In San Mateo and Santa Clara counties as 
sites for the placement of computer terminals. The, sites included the 
Redwood City Public Library, the San Mateo County Library, the Santa Clara 
County Library, and the San Jose Public Library. All libraries in the CIN 
network were encouraged to participate In the experiment by either directly 
referring patrons or by forwarding patron requests to one of the above 
I Ibrarles . 
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The Individuals who performed the on-line searching and search-related 
Interviewing had (with rare exceptions) no previous experience with 
on-line searching (1). A core of eight librarians (two from each library) 
received the standard Lockheed introductory course on DIALOG (2 days) and 
were allowed time to practice searching. These librarians then provided 
the search training to other staff members of their respective libraries. 

The experimental nature of this project differentiates It from the 
normal search situation. During the first year pf the project, each of the 
four participating libraries received both 16 hours per month of free 
scorch time and 16 hours per month of free demonstration time. Since the 
search time was free, searchers were able to experiment with the system 
while conducting patron searches. They were not under any pressure to keep 
the searches short and efficient. On-line search times, and subsequently 
connect costs, reported in this paper may be somewhat higher than they 
micjht have been If this were not an experimental project in 
which free search time was available to the libraries. 

PREVIOUS STUDIES 

The development of adequate statistical (time, cost) monitoring 
mechanisms In on-line systems has been a slow process. Monitoring programs 
took a large leap forward with the advent of commercial systems which 
require the precise recording, of user-system Interaction time for accurate 
bM-M-ng purposes. 

Monitoring of the user-system Interaction can take many forms. The 
simplest measurements deal with elapsed search time, .time spent searching 
particular data bases, and frequency of command utilization. One of the 
earliest reports on user hehavior as monitored by a retrieval system was 
presented by Summit (1969). Data reported included elapsed search time, 
number of Index terms used In a search, number of Boolean expressions used 
in a search, and number of citations printed by the system. Subsequently, 
there have been a number of other analyses. Of particular note Is a study 
by Benenfeld et al. (197*0 which reported a comprehensive set of 
characteristics of on-line searches. Among the variables computed were the 
time required to discuss the search with the patron, the search time at the 
terminal, the number of citations printed out, and the total user cost of 
the search. Benenfeld's experience indicated user costs per search In the 
range of $28 to $56 depending on the data base used. Lawrence, Well, and 
Graham (197*0 also gathered cost data on bibliographic searching and Flman 
(197U) surveyed some of the previous studies and presented his own 
computations Indicating that an average on-line search cost $47* Another 
aspect of on-line search costing that has received some attention In the 
literature Is the cost of operating the computer equipment. Lancaster 
(1973) reviewed some of this literature. It Is apparent that this area of 
cost analysis needs considerably more Investigation before the Internal 
economies of on-line searching cah be settled. 



(1)1 t Is only fair to note that one of the authors of this paper was also 
one of the searchers being studied , an employee of San Mateo County 
Library. We hope that no bias resulted. 
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METHODOLOGY 

The procedure for obtaining the data from which to compute the cost of 
on-line bibliographic searching required the cooperation of all Individuals 
engaged tn processing the on-line search requests. A set of seven tasks 
wns defined Including reference Interview, originating library preparation, 
DIALOG library preparation, search, DIALOG library follav-up, originating 
library follow-up, and fol low-up wl th patron. A form was then 'developed to 
collect data describing the amount of time spent on each task, the date on 
which the task was performed, and the Individual who performed the task 
(Appendix A). This time sheet traveled with the search request through Its 
processing. As completed time sheets were received, they were numbered and 
coded for computer processing. In order to protect the anonymity of the 
library employees performing the various tasks, employee names were 
replaced with code numbers for all computer tabulations. In the tables 
that refer to individuals performing search' tasks , code numbers will be 
used. 

Costs were developed both for each search performed and for each task 
within a search. Each task could be performed by a different Individual. 
In order to arrive at the direct labor cost for a search, It was necessary 
to multiply the pay rate for each Individual Involved In the search by the 
number of minutes each spent at a particular task. 

In addition to the labor cost of a search, two other costs must be 
considered. The first Is the fee paid to the commercial search service (in 
this case, Lockheed) to access their data base. This fee Is a function 
both of the length of time one Is connected to the system via telephone 
line and of the particular data base (such as ERIC, MTIS, Psychological 
Abstracts, etc.) that Is being searched. For each request, multiple data 
bases may be searched. In computing the on-line cost for a search, the 
cost per hour of accessing each data base was multiplied by the connect 
tine to arrive at a total data base connect cost. 

The second non-labor cost element Is the number of bibliographic 
citations printed at the commercial vendor's computer center and sent by 
mail to the searcher's location. A charge Is made for these off-line 
prints and this adds to the total cost of the search. 

Several other factors could be Included In the cost of a search. The 
major omission of this study was the cost of telephone Hne charges. Since 
all the libraries were relatively close to Lockkheed's computer center, 
they dialed directly to the center without the use of an Intermediary 
service such as TYMSHARE. If an Intermediary service had been necessary, 
these charges would have to be included In the total search cost. 

Other cost elements that were excluded from the study were terminal 
rental costs, physical space charges, and Indirect costs of overhead, 
administration, supplies, etc. Basically, the cost figures presented here 
summarize the direct costs of on-line searching. 
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The data for this study were collected during February, March,,, and part 
of April of 1975. Under the terms of the National Science Foundation 
grant, DIALOG service was provided free to the public for tKe first year of 
the experiment, and at a reduced rate for another year. The. three month 
study period was one which occurred six months after the project had begun 
and during which the service was still free to all users. 

It should be emphasized that the data comes from self-reports of the 
library personnel. Such a method Is of course less accurate than an 
outside measure, but was the only practical way to collect the amount and 
variety of data needed. 

At the end of the data collection period, Ml usable tlmesheets had 
been collected from the four DIALOG libraries (Table 1). Of these 411, 
33.6$ were from Redwood City Public Library, 25.1% from Santa Clara County 
Library, 22.6% from San Mateo County Library, and 18.7% from San Jose 
Public Library. Branch libraries and other CIN libraries took requests and 
relayed them to the DIALOG libraries; 16% of the requests during the 
reporting period showed some participation by these other libraries. 



THE REQUEST PROCEDURE 



A general outline of the DIALOG request procedure Is flowcharted In 
Figure 1. For the data collection, the process was divided Into seven 
tasks which formed the basic reporting units on the time sheet (Appendix 
A). 

The tasks were defined by both where and at what stage of the search 
process the activity took place. A maximum of two possible types of 
libraries participated in each search: a library with a terminal (called 
DIALOG library) and a cooperating library, a branch library, or other CIH 
member (called originating library). The tasks were further divided 
according to whether the activity occurred before or after the actual 
on-line search. The DIALOG search Itself was treated as a separate task. 
The tasks were defined as follows: 

REFERENCE INTERVIEW: time that the user spent negotiating the 
request by submitting and discussing It with a library staff member. 
This could take place at either a DIALOG library or an 'originating 
library. 

ORIGINATING LIBRARY PREPARATION: If the reference Interview was at 
a cooperating library, the staff there might spend some time recording, 
researching, and transmitting the request. 

DIALOG LIBRARY PREPARATION: time spent on the request at the DIALOG 
library with no patron present and before performing the on-line 
search. Typical activities Included recording the receipt of 
the request, filling out forms, and researching search terms. 

DIALOG CONNECT TIME or ON-LINE SEARCH TIME: the actual DIALOG 
search. 




9 

ERLC 



49 



7 




DIALOG LIBRARY FOLLOW-UP: time spent after the on-line search, with 
no patron present. Typically this would Include filling out reporting 
forms, examining search results, and notifying the patron or 
originating 1 Ibrary. 

ORIGINATING LIBRARY FOLLOW-UP: analagous to DIALOG library follow-up, 
but at a cooperating library. 

FOLLOW-UP WITH PATRON: at either type of library, the staff might 
spend time with the patron explaining the results of the search. 

The only one of these activities that was essential was the actual 
on-line search. Any combination of the other activities might be 
performed, depending on the nature of the request and on the library or 
libraries involved. The Individual libraries varied this general form In 
accordance with their own needs and policies. 

The Santa Clara County Library Is a large system with branches all over 
the county and a research library In Cupertino where the terminal was 
located. Users were free to submit requests through branch libraries or at 
the Cupertino library. Over the course of this study, virtually all 
requests originated at Cupertino. The requestor, on entering the Cupertino 
library, was directed to a Dl ALOG-tralned staff member, If possible, who 
performed the reference Interview. The librarian asked the patron If 
she/lie wished to be present during the search, In which case an appointment 
was arranged for some time in the future. Scheduled searches were limited 
to three or four per day, due to the demands of other staff dut'les. If the 
user did not ask to be present, the search was performed at the searcher's 
convenience, often but not necessarily by the same person who had performed 
the reference Interview. The user was cal led el ther when the on-line 
search was finished or when the off-line prints were received. Whenever 
possible a DIALOG staff member discussed the results with the patron. 

The Redwood City Public Library consists of, a main library and two 
small branches. Users with reference questions requiring the facilities of 
the main library were sent there directly, as were DIALOG requestors. When 
possible they were met by a Dl ALOG-tralned reference librarian, who 
performed both the Interview and search. The user was then called to come 
in to pick up the results and was met by a reference librarian If possible. 
Occasionally, the results were mailed to the patron. 
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The San Jose Public Library serves a metropolitan area. The users with 
questions on DIALOG were referred to the main library's science/business 
department whose personnel Included -two searchers. The other staff 
members in the department were also familiar with DIALOG. The user filled 
out the request form, and might have been briefly questioned by the staff. 
The primary searcher during the data collection period was a student 
volunteer who came In part-time to perform searches. No particular effort 
was made to have the patron talk with her, although she telephoned the 
requestor If she needed clarification of his/her request. The patron 
was notified, when the results were ready, by the searcher who explained 
the results, and the results were left in the science/business department 
to be picked up. A note was attached to the results encouraging the 
requestor to talk with the searcher If s/he did not understand the 
citation or was interested In obtaining specific articles. 

The San Mateo County Library alone among the DIALOG libraries derived a 
substantial portion (about (>0%) of Its requests from other libraries. Like 
the Santa Clara County Library, users were allowed to place^requests with 
the main library or with a branch or other CIN library; unlike Santa Clara, 
many users opted for the remote access. A librarian, although not 
necessarily one very familiar with DIALOG, general ly. performed a reference 
interview for each request. Searches received from other libraries or from 
walk-In business were placed with all other pending searches, to be 
performed at the librarians 1 convenience. The results were often sent to a 
branch library, even when the request originated at the main library. 
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Table 1 



Frequency Distribution of Requests by Dialog Library 



Library 


Number of 
Requests 


Percent Distribution 
of Requests 


Redwood City IKL; 


138 


■ 33.6 


Redwood City 
Menlo Park 1 


137 


33.3 


1 


0.2 


Santa Clara County (SC) 


103 


25.1 


Cupertino Research 
Center 


95 


23.1 


Gil roy 
Others 1 


3 


0.7 


5 


1.2 


San Mateo County (SM) 


93 


22.6 


Central Library 


37 


9.0 


At her ton 


20 


4.9 


Belmont 


6 


1 5 


Foster City 


3 


0.7 


Mill brae 


2 


0.5 


Woodside 


1 


0.2 


Public Health, 
and Welfare 


20 


4.9 


Other 


4 


1 ".0 


San Oose Public Library(SJ) 


77 . 


18.7 


Total 


411 


100.0 



An independent library that sends its requests to the indicated 
Dialog library 
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FLOWCHART OF DIALOG REQUEST PROCESS 





STAFF 

One major decision for a library Instituting this kind of service Is 
that of how to staff the service. The libraries considered two major 
al ternati ves. 

First, they could designate one or two people to do all the searches. 
The advantage to this would be that the searchers would quickly become 
highly skilled. Also, the responsibility for DIALOG and DIALOG-related 
activities would be clearly assumed. However, It would mean that at times 
the search staff would not be available to users with questions or 
problems. 

The alternative was to have all of the reference personnel add DIALOG 
searching to their usual repertoire of reference skills. This would 
diffuse the searching, and therefore diffuse both the experience and the 
responsibility. However, It would increase the availability of trained 
personnel to users so that a request could be handled efficiently by 
virtually anyone at the reference desk. 

Three of the libraries opted for the second alternative. At San Jose, 
however, due to special circumstances (l), most of the searching eventually 
devolved on one person. 

All of Redwood City's staff who participated In DIALOG performed, at 
one time or another, almost every step of the procedure, including the 
on- 1 1 no scorch. Generolly, a librarian followed one request through from 
beginning to end. Santa Clara County shewed roughly the same pattern, 
although they did have a few requests that originated at cooperating 
libraries so that the people who dealt with the patron did not perform 
searches . 

At San Mateo County Library, staff members from ether county system 
libraries frequently conducted some of the search tasks. Of the twenty 
people who were llsned on time sheets, only five performed searches, and 
only four could be characterized as reyular searchers. 

Table 2 suronarlzes the salaries of the 37 employees Involved In 
processing the on-line search requests. Participants 1 mean salaries for 
the job title ranged from a low of $658 per month for three library 
assistants to a high of $1325 for a supervisory librarian (2). For future 
reference, the table also shows the employee code numbers (Instead of 
employee names) for each Job title within each library. 



(1) Near-by San Jose State University threatened to swamp San Jose Public 
Library with requests, so an arrangement was made whereby a library school 
student volunteer did a large proportion of the searching. 

(2) Emp1oyees of each of the cities and counties in the study worked a 
different number of hours per week. Redwood City employees worked 38 hours 
per week while employees of Santa Clara, San Mateo and San Jose worked kQ 
hours per week. When labor costs were computed this factor was considered. 
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Table 2 

Salary Schedule for Library Employees 
Involved in Dialog Process 



Library 


Job Title 


Mean Monthly 
Salary 


Employee Code Numbers 
in this Classification 


Number of Employees 


Redwood 
City 


Librarian I 
Librarian II 
Librarian III 

k_ 1 k/ 1 U 1 IUII AAA 


$1,000.50 
1 ,083.00 
1 178 50 


24, 25, 27 
26 


3 
1 


banta 
Clara 


Libr. Assist. I 
Librarian I 
Librarian II 
Regional 

Librarian I 
Supervising 

Librarian 


764.00 
974.50 
1,074.50 

1,128.50 

1,306.00 


40 

35, 36, 37 
38, 41, 43 

42 

44 


1 

3 
3 

' 1 


San 
Mateo 


Libr. Assist. I 
Libr. Assist. II 
Librarian I 
Librarian II 
Librarian III 
Supervising 
Librarian 


658.00 
801 .00 
992.00 
1,112.50 
1 ,232.00 

1,325.00 


2, 9, 11 
13 

1 ,10,12,74, 16, 17,18 
6, 7,' 8 
4 

15 


3 
1 

. 7 
3 
1 

1 


San 
Jose 


Librarian I 
Librarian II 


955.00 
1 ,052.50 


50*, 5'i, 53,, 56 
52 54, 55 


4 ' 
3 



*Unpaid volunteer; salary calculated at Librarian I level for purpose of "study 
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In Table 3 data arc presented on the frequency with which various 
employees performed each of the seven tasks connected with a search. For 
the entire experiment, 1595 tasks were performed for the k]\ search requests 



averaging about 3*6 tasks per search. ^29 searches were performed for the 
411 requests Indicating that very few duplicate searches were made for each 
request* Employee 35 at Santa Clara County performed the greatest number 
of tasks while employee 23 at Redwood City ranked second. Together these 
two employees accounted for 20% of all tasks performed. 

DATA BASES 

One possible source of variation among the libraries was the choice of 
data bases used for the search. Since the different bases cover different 
subjects, variations in base use could Indicate that the libraries receive 
different types of requests, and/or that the librarians 1 search habits 
differ. Also, since the bases 1 hourly rates vary, differentiated use could 
affect the cost of searching (See Table 4). 



The most commonly used bases (1), for all the libraries taken together, 
were NT IS (accounting for 17*84% of the bases used), Psychological 
Abstracts (16.57*), and ERIC ( 13 • 35%) (Table 5). NTIS Is a highly varied 
base that covers a wide range of topics, and the documents abstracted were 
readily available to the project participants through purchase from NTIS or 
on loan from the California State Library. Pyschologlcal Abstracts (PA) 
and ERIC cover psychology and education, respectively. All three of these 
could be expected to be of use to many public library patrons. The next 
most used bases were Social Science Citations Index (9.51%), COMPENDEX 
(COMPutcrlzed Engineering InDEX; 9.29%), and Chemical Abstracts Condensates 
(8.23%). The first ties In well with the first three bases; the latter two 
are equivalent to well-known hard-copy services. The least used bases were 
all Predlcasts bases, including PATS Source (.11%), PATS Claims (.21%), and 
PATS Chemical and Electronic Market Abstracts Weekly (.85%). All are 
special-purpose bases. 

Considering the libraries Individually, their data base usage did 
differ from the group percentages, but not markedly. For example, the 
group proportion for NTIS Is 17.84%; Redwood City's figure was 19% as were 
Santa Clara's and San Mateo's; while San Jose's was 13%. Psychological 
Abstracts' group percentage was 16*57%; the Individual proportions were 
18%, ]kZ 9 22% and 13%. 



(l)More than one base may be used on a search; these are the proportions of 
bases used, not of searches performed* 
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Table 4 



Data Base Connect Charges Per Hour 



Data base name 


Charge per connect hour 


Charge per off-line print 


ERIC 


$ 25 


$ 0.10 


CHEM ABSTRACTS 


45 


0.10 


EXCEPT. CHILD ABS 


25 


0.10 


NTIS 


35 


0.10 


SSCI 


70 


0.10 


COMPENDEX 


65 


0.10 


AIM-ARM 


25 


. 0.10 


NAL/CAIN 


25 


0.10 


PSYCH ABSTRACTS 


50 


U. 1 U 


INSPEC-PHYSICS 


.45 


0.10 


INSPEC-ELECT. EN 


45 


0.10 


INSPEC-COMPUTERS 


45 


0.10 


ABI/ INFORM 


65 


0.10 


PATS CMA/EMA 


90 


0.20 


PATS C/EMA WEEKLY 


90 


0.20 


PATS F and S 


90 


0.20 


PATS Source 


90 


0.20 


IFI/Claims 


$ 150 


$ 0.10 



These were the bases available during the data collection period. 
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Table 5 
Data Base Usage by Library 



Data Base Name 


Number of Data Base Uses by Library 


total 
Uses 


Percent 
Distribution 
of Uses 


Redwood 


Santa 

Tiara 


San 

Ma 4* on 
no leu 


San 

tine o 


ERIC 


44 


38 


18 


25 


125 


13.35 


CHEM. ABSTRACTS 


29 


15 


21 


12 


77 


8.23 


EXCEPT. CHILD, ABS. 


2 


3 


5 


6 


16 


1.71 


NTIS 


61 


45 


35 


26 


167 


17.84, . 


SSCI 


32 


27 


5 


25 




Q 51 


COMPENDEX 


24 


27 


5 


25 


89 


9.29 


A1M-AUM 


1 


1 


3 


5 


10 


1 .07 


CAIN 


14 


5 


12 


■ 9 


40 


4.27 


PSYCH. ABSTRACTS 


56 


33 


40 


26 


155 


16.57 


INSPEC-PHYSICS 


12 


7 


4' 


2 


25 


2.67 


INSPEC-ELECT.EN6 


4 


12 


2 


2 


20 


2.14 


INS PEC -COMPUTERS 


2 


4 


3 




■ 9 


0.96 


ABI-1NF0RM 


20 


11 


11 


20 


62 


6.62 


PATS CHEM ELECT 


7 


10 




6 


23 


2.46 


PATS CHEM-EL WEEK 




3 




5 


8 


0.85 


PATS F and S 


10 






10 


20 


2.14 


PATS SOURCE 


1 








1 


.11 


CLAIMS /PAT PRES 


1 






1 


2 


.21 


TOI AL ~ " 


""320 


243 


180 


193 


936 




Present 
Distribution 


34.19 


25.96 


19.23 


20.62 




100.00 
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Although on the library level no significant differences appear, when 
broken down further to show the Individual searches, differences become 
apparent (Tables 6 and 7). The most skewed figures are for searcher 3> who 
Is a special librarian and whose search pattern reflects the specialized 
Interests of her clients. The other searchers, however, have no such easy 
explanation for their preferences. Looking at the most frequently used 
base, MTIS, the frequency of use varies from 6.4$ to 23.8$ (Table 7). Even 
excluding searcher 3, Psychological Abstracts varies from a low of k.S% all 
the way up to a high of 30. 8$. 

We can only guess at reasons for this variation. It may be that some 
searchers have certain subject expertise, and that the searches within a 
library are parcelled out accordingly. However, none of the libraries 
reported any such pattern to their searching. It Is more likely that users 
fall into habits of searching and tend to use bases with which they are 
familiar. The most frequent searchers (23 and 50) show more of a 
scattering among the bases, so It may be that their added experience has 
encouraged them to branch out more, while other searchers stick with "safe n 
bases. Also, doing more searches will no doubt bring the searcher a 
greater variety of topics, and require a greater variety of bases. These 
are just conjectures, as the figures are not conclusive. I f the searchers 
were either very conservative In their use of "new" bases or were strictly 
segregated by subject, they would center their searches on a handful of 
bases. However, almost all of them used each of the more popular bases at 
least once. 

Table 8 breaks down the average tlrife spent on each data base by the 
most active searchers. Again, the resMlts show very little uniformity 
among the searchers. No one base tends to be a time-consuming base. The 
searchers varied widely in the average, time that they spent on a single 
base. There seems to be some lntra-1 Ibrary similarity, but It Is very 
slight; the San Mateo County Library searchers all had fairly low averages, 
the Redwood City searchers a little higher, the San Jose people about the 
same as Redwood City or perhaps a little higher, and Santa Clara County 
highest of all. These figures are not directly comparable with the mean 
on-line time per search, since any number of bases can be used on a single 
search. However, they are something to keep In mind for the discussion of 
time per task that follows. If a search used the same number of bases, 
and the average time per base differed among the libraries, the average 
on-line time per search should vary accordingly. 

OFF-LINE PRINTS 

The number of off-line prints requested was a major variable In the 
cost of the search. San Mateo County tried not to print more than fifty 
citations per search, but that was not a hard and fast rule. There were 
two reasons for this guideline. First, the number of prints available 
under the terms of the grant was limited (although no library overran this, 
limit during the first year of the study). Second, If the patron received 
too many prints with too much irrelevant Information, the precision and 
therefore the quality of the search was lessened. 
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Whatever Its effect on the quality of the search, San Matec's practice 
of limited off-line prints had a marked effect on searches completed at that 
library. San Mateo County printed an average of 29 citations per search 
off-line, whereas San Jose printed 66, Santa Clara County 71 f and Redwood 
City 75. 

Table 9 breaks down Into Intervals the nurrtber of citations printed out 
off-line per data base (more than one base could be used on each search) by 
searcher and Indicates the mean number of prints per base both for the 
searches for which prints were made (the non-zero entries), and for all 
searches, including those for which no prints were made. The figures 
differ strikingly. 

The Intervals show an expected decrease In the number of occurrences, 
moving from smaller nunbers of printouts to larger. Ho off-line prints were 
requested from more than half of the data bases; this could be because the 
search results were not satisfactory, or because the relevant citations 
were printed on-line. No searcher made a practice of printing monumental 
bibliographies. The mean values are more informative, showing some fairly 
significant differences among searchers. 

The mean number of off-line prints for all cases (Including those 
searches for which no prints were made) was U.26 for San Mateo County's 
searchers taken all together, ranging Individually from 10.73 to 19.0** 
(Table 9). Redwood City's overall average was 32.98; Its librarians ran 
from 16.9*» to 39.38. Santa Clara's overali figure was 29.^0, the 
Individual flyures from 21.81 to **0.39i San Jose's library-wide mean was 
23.92, Its Individual low 18.31 and Its high 32.37. 

Generally, It can be said that searchers varied widely when deciding 
how many citations were necessary and/or acceptable to the user. The 
intra-1 ibrary similarities are marked only for San Mateo County. Otherwise 
it was the Individual and not the library who was the determining factor In 
the number of prlnt-outs. 

TIME PER TASK 

Another major source of difference was the staff ,tlme devoted to each 
task. From the descriptions of the various libraries' request procedures, 
It Is evident that some placed more emphasis on certain tasks than did 
others. Tables 10 and 11 quantify this difference In two ways. Table 10 
Is the mean time per task, by library, for all non-zero entries only; I.e. 
once It Is decided that a task Is to be performed, that Is the average time 
for that task. Table 11 averages Into Table 10's figures the entries with 
values of zero, I.e. those searches for which that task was not performed. 
In many cases zero entries are a significant proportion of the 
observations. 
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Table 10 

Mean Time Per Task By Library 
(ir, minutes) 





Mean Task time By Library 
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Table 11 

Mean Time Per Task by Library, All Entries 
(in minutes) 





Mean Task Time by Library 
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Table 10 shows differences among libraries, but none vary much from the 
group mean. The single exception Is search time, which varies from a low 
of 1 A .09 minutes for San Mateo County to a high of 30.1*2 minutes for Santa 
Clara County, with an overall mean of 22.72 minutes. (More about this 
later.) However, Table 11 brings out more differences. For Instance, 
reference Interviews were around 6.3 minutes for all libraries except Santa 
Clara County, where the average was 11.61. The numbers of observations In 
the two tables explain the difference; 86* of Santa Clara County's requests 
were accompanied by a reference interview, while the next highest 
percentage was San Jose at 73*. Similarly, DIALOG library follow-up time 
as measured In Table 11 varies from a low of k.kj minutes (Redwood City) to 
a high of 1*t.56 minutes (San Mateo County). Only kP.% of Redwood City's 
requests had the DIALOG 1 Ibrary fdl low-up performed, while 89% of San Mateo 
County's did. 

The conclusion therefore Is that the determining factor In time per 
task was whether or not the task was performed. Once the decision was made 
to perform the task, the time requlr^J was similar among libraries. The 
search time was the one Important exception. This time Is especially 
Important since It determined not only salary expenditures, as did the 
other tasks, but also the charge for connect time, which, as will be seen, 
was a significant part of the overall cost. 

In order to statistically determine If there were differences In the 
times taken to perform each of the seven tasks across the four libraries, a 
set of Analyses of Variances was conducted. These variances are reported 
In Table 12. The table shows that there were significant differences In 
the times taken to perform the reference Interview, search, and DIALOG 
1 Ibrary fol low-up. 

While the Analysis of Variance Indicated some differences In group 
means, by itseif It did not i ndl cate whl ch of the four library's task times 
was significantly different from any other. To determine this, systematic 
comparisons of all possible combinations of task time mean values were 
performed and the results evaluated using Scheffe's test (1). Table 12 shows 
that for the Reference Interview there was a significant difference between 
Santa Clara's (SC) time and the times of Redwood City (RC) , San Mateo (SM), 
and San Jose (SJ) , but that there was no difference between the latter three 9 
meons. Similarly for search time, Redwood City and San Mateo were 
significantly different from Santa Clara and San Jose. 

PATRON'S PRESENCE 

One decision that the participating libraries had to make was whether 
the requestor should be present during the search. On the one hand, the 
requestor was often much more familiar with the subject than the librarian 
was and could provide both an Instant evaluation of how well the search was 
going and perhaps suggest alternative search strategies. The -process of 
the search might also help draw out a non- communicative patron, and help 
him/her understand what the librarian needed to know. 



(l)Sce (Kirk, 1968) for a discussion of this procedure. The test was 
conducted at the 0.05 level of significance. 
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Analysis of Variance for Task Times 
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On the other hand, a user unfamllPar with the search process could slow 
the search down, requiring lengthy explanations during the terminal 
sessions. The librarian might also prefer not to have someone watching 
over his/her shoulder during the search. 

Ultimately, only Santa Clara County made a regular practice of allowing 
the patron to be present during the search* Thus figures are available on 
the relative lengths of searches with and without the patron, but only for 
Santa Clara County. 

Of the 103 Santa Clara County searches reported, 60 were performed 
without, the patron, and 43 with the patron. The average search time for 
those with no patron present was 25*0 minutes versus 37*9 far those with 
patron present. The chance of this large a difference resulting simply 
from random fluctuations Is less than )%; searches with the patron take 
significantly longer. However, the total time required for all tasks did 
not vary that much; without patron, the average was 56*3 minutes, with 
patron It was C1.0 minutes. There is a 7*5% chance that the sample means 
are not significantly different. Thus It is not conclusive that the 
patron's presence at the search has an adverse effect on total search time. 
It may be that the patron's presence at the terminal shortens* the time 
required for either the reference interview or for the follow-up with the 
patron. 

COSTS 

All of these time figures can now be translated Into costs, using the 
salary and fee schedules (Tables 2 and 4). Rather than calculate costs 
based on the average time figures arrived at earlier, the procedure in this 
study was to calculate directly the actual costs for each search based on 
the time required, the salary of the person performing the task, and the 
data base(s) used. 

The mean salary costs for each task, by library, are presented In Table 
13* The figures for the Individual I ibrarles .cluster reasonably around the 
means for the group as a whole. The one exception Is originating 
library figures, for which only San Mateo County had enough observations to 
be meaningful. The costs were quite low, averaging $1.03 for the reference 
interview, $1.95 for originating library preparation, $1.02 for 
DIALOG library preparation, $1.16 for follow-up at the DIALOG library, 
$1.07 for originating library follow-up, and finally $,.74 for the 
post-search time spent with the user. 

Table 14 presents another analysis of the cost:, incurred In the on-line 
search Itself. The overall average was $17.29 for computer time, $9*16 for 
off-line printouts, and $2.24 for the labor at the terminal. This, 
combined with $5*02 for all other labor (detailed In Table 9) % brings the 
cost for the entire search process, for all libraries, to an average of 
$2fc.'»1. 
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Table 13 

Mean Salary Cost Per Task By Library 
(in dollars) 





Mean Task Cost By Library 
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Table 14 ' 

Search Costs 
(in dollars) 





Mean Task Cost By Library 
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Cost Element 
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City 
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County 


San Jose 


Mean 
Cost 


Data Base Charges 


$14.51 


$22.16 


$10.55 
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$17.29 


Off-Line Print 
Charges 


9.73 


12.50 


3.64 


12.96 


9.16 


Search Labor Cost 


2.03 


2.83 


1.47 


2.70 


2.24 


Labor Cost- for All 
Other Tasks 


3.64 


5.91 


6.22 


4.93 


5.02 


Total Cost of Search 1 


25.33 


35.17 


19.74 


35.19 


28.41 



l Thc total cost is not additive due to differences in the number of observations 
for each cost element. 




30 



The Individual libraries differed significantly from this mean, ranging 
from a low of $19. 7** (San Mateo County) to a high of $35.19 (San Jose) and 
$35.17 (Santa Clara County). Redwood City was a little below the average a 
$25.33. No one item accounts for the differences. 

San Mateo County, with the lowest overall average, had the highest 
labor cost for activity other than time at the terminal. They more than 
make up for this, however, by spending less on terminal time, both for 
computer time and labor, and also by printing far fewer citations. 
Referring back to the mean time per task in Table 10, It is evident that 
San Mateo County's searchers spent significantly less time at the terminal 
par search. The higher costs for other labor probably came from the added 
stop incurred when requests originate at a library other than the one 
at which the terminal is located. 

Santa Clara County and San Jose differed by only a few cents In the 
total search cost but their breakdowns were not Identical. San Jose spent 
more on data base charges, though a little less on salaries for the other 
tasks. This Indicates the use of more expensive bases and/or less 
expensive personnel (note that their most prolific searcher was 
the unpaid volunteer whose time was evaluated at the Librarian I rate). 
As Indicated In Table 10, San Jose generally spent a little less 
time on each task than did Santa Clara County. 

To look a little closer at the search process , Table 15 presents search 
costs for the high-frequency searchers. At this level there is more 
across- 1 Ibrary variation. The three searchers who spent the least salary 
at the terminal belong to the low-ranking San Mateo County (searchers 8, 3> 
and 10). The next three lowest are from other libraries, two from Redwood 
City (23 and 25) and one from San Jose (5?.). One of Redwood City's 
searchers (23) Is a relatively expensive Librarian Ml. The highest salary 
costs were incurred by Librarian 1 1 s from both San Jose (number 53) and 
Redwood Ci ty (26) . 

The data base charges, which are the largest single element In the 
search cost, follow a similar pattern. The six lowest searchers In terms 
of salary also account for five of the six lowest searchers In data bas« 
charges. The off-line print charges, as indicated earlier, differ 
strikingly, bringing all four of San Mateo County's searchers down to the 
four lowest total search costs. San Mateo County searchers are followed by 
searcher 23 from Redwood City and 32 from San Jose. The highest total 
search cost was attributed to searcher 37 of Santa Clara County, followed 
by searcher 36 from Santa Clara County. 

Generally, however, differences within a library were not as striking 
as those between libraries. Thus it appears that although Individual 
searchers do vary, the overriding consideration Is the library at which 
they are searching. This could be a result of library policy, either 
written or implied (evidence the San Mateo County effort to keep down the 
number of prints). It could, also be a result of Interaction among the 
searchers at a library, arriving among themselves at a definition of what 
Is an acceptable search length and number of prints. Also, since most of 
the searchers received at least part of their training from others In their 
own libraries, it could be that the Intramural similarities reflect a 
common teacher. 
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Table 15 

Mean Search Cost for Selected Searchers 
(in dollars) 
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24.86 


26 


3.43 
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10.97 


37.57 




50 


2.48 


22.07 


9.67 
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3.96 


34.98 
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Two factors determine the computer charges for connect time: the time 
required for the search and the choice of data base. It Is possible to 
separate out those two effects by considering the ratio of the cost 
of data base connect time to search time. This has the effect of 
normalizing the connect charge to reflect varying lengths of time 
spent searching. The actual ratios were: 

Redwood City $ .7^/min, 

Santa Clara Co* $ .73/mln. 

San Mateo Co. $ .77/mln. 

Son Jose $ .85/mln. 



From this It can be concluded that all libraries were spending roughly the 
same amount per connect minute for computer time. San Jose was slightly 
hlyher l denoting a slight tendency to make greater use of the more 
expcnsl ve bases , ♦ 

Similarly, the salary cost for any part of the search depended both on 
the time spent and the salary of the person performing the task(s). 
Considering again the on-line search, the ratios of labor cost to time at 
the terminal for the libraries were: 

Redwood City $ .10/mln. 

Santa Clara Co. $ .09/min. 

San Mateo Co. $ .11/mln. 

San Jose $ .10/mln. 

The libraries did not vary greatly in the overall salary cost per minute 
at the term! nal . 

Broadening this analysis to the entire DIALOG process, the ratios of the 
total cost of the DIALOG process (labor, computer charges, printouts) to 
the total time, from reference Interview through follow-up were: 

Redwood CI ty $ .71/mln. 
Santa Clara Co. $ .55/mln. 
San Mateo Co. $ .33/mln. 
San Jose $ .68/mln. 

A likely explanation for the low San Mateo County figure Is that their 
searches tended to be more labor-intensive, while the others were more 
computer- Intens Ive and computer time Is much more expensive than 
people-time. Redwood City's searches also tended to ** * less computer 
time. However, their off-line printout charge was muc higher than San Mateo 
County^ and their chief searcher during the data collection was a 
relatively expensive Librarian Ml (the other libraries relied mainly on 
Librarian l's)0). 

(l)San Jose Public Library Kas recently Indicated that with the advent of 
user charges for computer time, they are moving toward a more 
labor-intensive and less computer- in tensive search pattern. 
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COMBINATIONS OF TASKS 

In the description of the DIALOG process as It Is performed In each 
library, It was pointed out that the libraries differed In the emphasis 
placed on different tasks. For Instance, San Jose made no special effort 
to perform either a reference Interview or to spend post-search time with 
the patron. Santa Clara County, on the other hand, tried to have the 
patron submit to a reference Interview with a DIALOG staff member. 

The greatest number of tasks that might possibly be performed would be 
for a search that originated In a library without a DIALOG terminal, for 
which all seven tasks listed on the time sheet could be performed. For a 
request originating at a DIALOG library, the maximum number of tasks would 
be five, eliminating the two tasks that are specific to non-DI ALOG 
libraries. 

Table 16 shows the twelve (out of 128 possible) combinations of tasks 
that were most frequently performed. These twelve accounted for 338 of the 
searches performed, or 825$. The most used single pattern of tasks was 
number ten In Table 16. This pattern consisted of a reference Interview, 
DIALOG preparation, search, DIALOG follow-up, and a follow-up with user; 
the "complete* 1 Dl ALOG-orl g Inat I ng pattern. The next most common was 
pattern seven, the same as ten but with the DIALOG preparation omitted. 
Pattern twelve, every possible task performed, occurred only ten times. 

Given that different patterns of tasks mean that more or fewer steps 
ore performed for each search, It follows that different patterns 
result In different costs. 

Table 17 shows the mean total search cost, by library., for each of the 
frequently occurring combinations of tasks. Surprisingly-, the lowest 
overall mean Is the one with the most tasks (combination number 12). This 
figure Is not conclusive, however, due to both the low number of cases 
(ten) and the fact that all cases were from San Mateo County, which had 
consistently low search costs. 

The second least costly (again with figures only from one library, In 
this case Redwood City) was the sixth Instance, which consisted of a 
reference Interview, the search, and follow-up with patron. 

Interestingly, the bare bones search - no task performed other than the 
actual on-line search • although Inexpensive, at $15.67 (number one), was 
only the third least expensive combination. 

The most expelve combination, considering all libraries together, was 
combination number eight, reference Interview, search,. DIALOG library 
follow-up and follow-up with user at $38.08, The next most expensive was 
pattern number ten, which Includes all the tasks In combination eight plus 
DIALOG library preparation. The additional task lowered the cost to $3^,23. 
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Table 16 

Most Frequent Combinations of Tasks Performed for DIALOG Searches 





Combination of Tasks 


Frequency 
of 

occurrence 


ation 

11 UlliUt. I 


Reference 
Interview 


Originating 

Library 
Preparation 


Dialog 
Library 
Preparation 


Search 


Dialog 
Library 
follow-up 


Originating 
Library 
follow-up 


Follow-up 
with User 


1 


0 


0 


0 




0 


0 


0 


21 


2 


0 


0 


0 


! 


1 


0 


0 . 
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Note: ,0 indicates task not performed; 1 indicates task was performed. 
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Table 17 

Total DIALOG Search Cost for Frequently Occuring 
Combinations of Tasks 
(in dollars) 
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These datn on tnsk combinations Imply that there Is no direct 
correlation between the number of tasks performed and the cost of the 
search. San Mateo County had the lowest average cost, while generally 
performing more tasks per search than other libraries did. 

ELAPSED TIME 

Two measures of time are Important to the user of any kind of search 
service. One is the time that the user or his/her surrogate actually 
spends on the search. The use of on-line searching reduces this measure 
from hours, even days, which must be spent leafing through abstracting and 
Indexing publications, to only minutes spent conferrl ng wl th the search 
personnel . 

The second measure Is waiting time - how long the user must wait from 
the Instigation of the request until the results are received. The waiting 
time Is an Indication of'how quickly the search service processes the 
request. 

The mean time that requests spent In the system, (1) from the first 
patron contact until the last task (whatever that was) for all of the 
libraries together was 7.79 days "(calender t not working days). For the 
Individual libraries the figures were: Redwood City, 6.02 days; Santa Clara 
County, 87 days; San Mateo County, 1^.67 days; and San Jose, 6.79 days. 
(2) 

Table 18 ■Hwilyziss this data further. This table shows the mean time 
between various pairs of tasks, by library. Not every task was performed 
for each search; the figures are only for those cases In which both 
elements of the pair In question were performed. It Is also worth noting 
that the pairs examined were not necessarily successive tasks, but those 
for which a significant number of observations were available. There Is 
overlap among the pairs examined, e.g. reference Interview to search 
subsumes reference Interview to originating library preparation, reference 
interview to DIALOG library preparation, and DIALOG Mbrary preparation to 
search. 

Redwood C 1 ty 1 s data show a regular progression through the tasks. 
Reference Interview to search required about two days of the six overall; 
search to fol low-up wl th patron tended around three days. Considering that 
off-line printouts were sent first class mail, and required a minimum of 
two days to arrive, the figures were quite good. 



(1) The minimum time possible Is one day and that is for same day service. 

(2) A1I of the libraries would expedite rush requests, If the patron had a 
good reason. 
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Table 18 
Days Elapsed Between Tasks 





Library 


M 
E 




KeaWOdd City 


banta 


Llara 


ban Mateo 


San Jose 


Task Interval ' 


Mean 


* of 
obser- 
vation 


Mean 


H of 
obser- 
vation 


Mean 


# of 
obser- 
vation 


Mean 


# of 
obser- 
vation 


A 

N 


Reference Interview to 
Originating Library 
Preparation 










1.74 


14 






1.74 


Reference Interview to 
Dialog Library 
Preparation 


1 .91 


24 


2.23 


38 


4.63 


52 


5.46 


47 


3.90 


Originating Library 
Preparation to Dialog 
Library Preparation 






4.50 


2 


4.94 


37 






4.9 


Dialog Library 
Preparation to Search 


.59 


41 


88 


45 


3.77 


76 


0.75 


58 


1.79 


Reference Interview to 
Search 


1 .93 


92 


3.05 


89 


8.77 


58 


6,12 


56 


4.9 


Search to Dialog 
Library Follow-Up 


1.60 


66 


0.16 


92 


1 .85 


83 


1 .08 


67 


1 .19 


Search to Originating 
Library Follow-Up 


2.00 


2 


2.50 


2 


7.31 


38 






6.83 


Search to Follow-Up 
With Patron 


3.03 


30 


2.66 


66 


7.72 


43 


3.03 


31 


4.00 


Dialog Library Fol low- 
Up to Originating 
Library Fol low-Up 










fi Of) 


JO 






6.00 


Originating Library 
Fol low-Up to Fol low- 
Up With Patron 










0.93 


15 






0.93 


Dialoy Library Follow- 
Up to Fol low-Up With 
Patron 


1.70 


20 


3.70 


60 


7.34 


41 


2.26 


30 


4.14 


Average time in system, 
all cases 


6.02 


140 


4.87 


103 


14.67 


91 


6.79 


77 


7.79 
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Santa Clara County shaved about a day off Redwood City's time, 
primarily In the post-search time. The Santa Clara County off-line print 
figures showed a greater tendency on the part of searchers to request no 
off-line prints. This tendency might have been a result of having the 
patron present frequently (citations that might have been printed off-line 
in the patrons 1 absence might be printed on-line or not at all). This 
would eliminate the wait for the U.S. Postal Service and might account for 
Santa Clara County's lower post-search figure. 

San Mateo County was the slowest of the group, at 14.67 days. Table 18 
shows them with consistently higher figures than the other libraries, with 
the time divided almost equally between pre- and post-search (8.77 days vs. 
7.72). San Mateo County was the only library with a significant number of 
tasks performed at the originating library, which added a step and a 
possible bottleneck to the search process. It may also be that the lack of 
direct contact with the patron made San Mateo County tend not to hurry In 
searching a request once it was. re eel ved. Furthermore, DIALOG requests 
were competing with other subject requests received from the branches In 
much the some way as the DIALOG requests. In other words, Dl ALOG-requests 
simply had to wait their turn. 

San Jose's total of 6.79 days was a distant second to San Mateo County. 
The bulk of that time (6.12 days) was pre-search, echoing San Mateo 
County's pattern of having the request sit waiting. In this case, hewever, 
the wait was for San Jose's part-time searcher. Also, the .San Jose staff 
hod complained early in the project of being squeezed between DIALOG and 
their regular duties. This long pre-search wait and short post-search time 
suggest a general backlog. Items were handled speedily, once their turn 
came, but there were others competing for the staff's attention. 

CONCLUSIONS 

This study hes reported the costs of on-line bibliographic searching In 
four public libraries in the San Francisco Bay Area using the Lockheed 
DIALOG system. This study was conducted durfng a period when search 
time was provided cc the libraries without charge and the searchers were 
moderately experienced. It was found that there were seven different tasks 
involved In the search process. In the. course of the study 35 Individuals 
from the four libraries reported their own processing times for 411 search 
requests. 

Two limitations anouJd be considered in evaluating the results of this 
study. One Is the self reporting nature of the data. Participants 
recorded time spent on search tasks rather than being observed and measured 
by others. Consequently, a possible bias Is Introduced depending on the 
accuracy with which the searchers recorded their times. A second 
linltation Is the experimental nature of the project. On-line searching of 
the DIALOG system was provided at no cost to the public libraries through a 
grant from the National Science Foundation. Thus the searchers were under 
little economic pressure to-perform effectively. ' 



81 



39 




Considerable variation in the time" required to perform search tasks was 
found between libraries and between searchers. For example, the average 
connect time to a data base for an individual search was 22.7? minutes. 
But this varied from 19.63 minutes at the Redwood City Public Library to 
30. ^2 minutes at Santa Clara to 28.31 minutes at the San Jose Public 
Library. Similar variations were found In the overall cost of a 
bibliographic search: Redwood C I ty 1 s average was $25 • 3B t Santa Clara's was 
§35.17, San Mateo's was $19. 7^ San Jose's was $35.19i and the overall 
average for the 411 searches was $28.41. The most expensive portion of the 
overall cost was the actual on-line search time, which averaged $17.29 for 
all searches. The next most expensive element of the total cost was that * 
of off-line prints--this averaged $9.16 for all searches. 

The only element that was omitted from the cost calculations was 
telephone line charges. This was due to the fact that the four libraries 
were In close physical proximity to Lockheed's computer center > If this 
situation did not hold for other searching locations it would bs necessary 
to add the telephone costs. Assume a $10 per hour telephone connect charge 
through an organization such as TYMSIIARE, and a mean search time of 22.7? 
ninutes. This results In an average telephone charge of $3.79 which, when 
added to the $28.41 search total, totals $32.20 for the average search. 
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Appendix A 
DIALOG. Time Sheet 
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DIALOG TIME SHEET 

•Please enter time in minutes to the nearest five minutes. Please date and 
initial each entry. 

DIALOG LIBRARY: 1. RCPL 2. SCCO 3. SMCO 4. 



Patron name (optional): 
Request number: 

Originating Library 



EVENT 


TIME IN DATE INITIALS 
MINUTES 


REFERENCE INTERVIEW 
initial patron 
contact 




ORIGINATING LIBRARY 
PREPARATION TIME 
without patron 




DIALOG LIBRARY 
PREPARATION TIME 

u-fHinnf- patron 




DIALOG CONNECT 
TIME 


don't enter time (do date and initial) 


DIALOG LIBRARY 
FOLLOW-UP TIME 
" without patron 




ORIGINATING LIBRARY 
FOLLOW-UP TIME 
without patron 




FOLLOW-UP TIME . 
vith oatron 




MISCELLANEOUS 
describe: 





DIALOG CONNECT TIME Patron present? Check if yes 



DATA BASE ELAPSED TIME NO. OF OFF-LINE PRINTS 



Use reverse to list add! Clonal bases searched on the same request as above, 
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Appendix B 



Alternative Cost Calculations 



In Tables 13 and )k of the— text the mean salary costs, and mean search 
costs were summarized* These tables were calculated on the basis that each 
occurrence of a task resulted in a cost of performing the task and that the 
total cost was the sum of the task costs for all tasks which were 
performed. 

Tables B-l and B-2 are the analogies to Tables 13 and 14 with a 
different method of computation used. In Tables B-1 and B-2 the mean unit 
cost of a task is determined by dividing the total cost by the total number 
of valid observations, not just the nunfcer of observations for which the 
time was non-zero* V/hile the mean search costs (Table B-2) remain 
relatively stable, there is a considerable reduction in the salary costs 
per task (Table B-1), 
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Table B-l 



Mean Salary Cost Per Task by Library, 
All Entries 

(in dollars) 





Mean Task Time by Library 


Task 


Redwood 
City 


Santa Clara 
County 


San Mateo 
County 


San- 
Jose 


Overall 




Mean 


Mean 


Mean 


Mean 


Mean 


Reference 
l ntervi ew 


.68 


1.05 


.65 


.58 


.75 


Originating 
1 ibrary 
preparation 






.95 




.22 


Dialog Hbrary 
preparation 


.39 


.46 


.81 


.64 


.55 


Search 


2.03 


2.83 


. 1.47 


2.70 


2.24 


Dialog library 
fol low-up 


.42 


1.04 


1.44 


.75 


.87 


Originating 
1 ibrary 
follow-up 






.44 




.11 


Follow-up 
with patron 


.18 


.48 


.37 


.24 


.34 
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Table B-2 

Mean Search Costs, All Cases 
(in dollars) 







Mean Task Cos 


t by Library 




Overall 


Cost Element ' 


Redwood 
City 


Santa Clara 
County . 


San Mateo 
County 


San 
Jose 


Mean 
Cost 


Data base 
charges 


14.51 


22.16 


10.55 


23.69 


17.29 


Off -Line 
Print 
charges 


7.48 


7.09 


2.90 


6.56 


6.17 


Search 
Labor 
Cost 


2.03 


2.83 


1.47 


2.70 


* 2.24 


Labor Cost 
for All 
Other 
Tasks 


1.67 


3.03 


4.66 


2.21 


2.81 


Total Cost 
of Search 


25.69 


35.11 


19.58 


35.16 


28.51 
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